ci: wire the DEV-2203 suites — post-deploy smoke, weekly canary, monthly matrix - #189
Merged
Conversation
…hly matrix e2e-live.yml grows three ways in: the manual dispatch it always had (now with ai and pkg_pr_new_ref inputs), a weekly Monday prod canary for external drift (the hosted bundler, npm, the broker, docs-bucket rot — deploy breakage is caught at deploy time, so daily would only buy triage), and a workflow_call smoke mode both deploy workflows now invoke after their curl checks: the @smoke subset, one container, minutes. The full deployed run folds in the specs nothing ever ran (row-striping, preview-recovery's live half, import-live) plus the DEV-2203 additions, split by what they hold: the sandpack/API-read step runs parallel, the container step strictly --workers=1 against the global pool of five, the authed share round-trip treats a rotten E2E_BROKER_TOKEN as a notice rather than a red run, and the AI step runs on the canary or by asking. e2e-starter-matrix.yml gains the monthly sweep (1st, 05:00 UTC) — the only coverage of the full starters × majors surface, at a cadence matched to npm-release timescales. Concurrency keeps deployed runs uncancellable so no run ever strands container sessions.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
added 2 commits
August 17, 2026 08:37
The post-deploy smoke boots a live-preview container; an unconditional cancel-in-progress killed the whole run — smoke included — on the next push to master, stranding that session in the global Sandbox pool (cleanup lives in a finally a cancellation never reaches). Deploys now queue, matching deploy-runner-api.yml. The other #189 finding — e2e-live listing style-panel.spec.ts before it existed on this branch — is resolved by linearising the stack: the spec now sits in this branch's ancestry (185 → 186 → 187 → here).
added 2 commits
August 17, 2026 08:45
…ugbot #189) The authed-share step puts a live session JWT in sessionStorage and an Authorization header; a Playwright trace records both, GitHub does not redact secrets inside artifact zips, and this repo is public — so a failed canary would have published a valid @handsontable.com session for seven days. That step now runs with --trace off --retries=0 and scrubs test-results/ and playwright-report/ before the on-failure upload can see them (|| status capture: Actions runs bash -e, so a plain $? after a failing command is unreachable). The monthly starter matrix moves to 03:00 on the 1st: at 05:00 a Monday-the-1st would start it alongside the weekly canary, and matrix --workers=2 plus the canary's serialized container suite against the same global pool of five is exactly the contention both workflow headers warn about.
…to master.yml #184 replaced the two deploy-runner-*.yml workflows with the master.yml orchestrator, so this branch's smoke jobs — appended to those deleted files — become one smoke job in the orchestrator instead: the @smoke subset runs once after whichever deploys fired, rather than once per deploy workflow. AGENTS.md's table merges both descriptions.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ef126e9. Configure here.
…rkflows # Conflicts: # .github/workflows/e2e-live.yml
demtario
approved these changes
Aug 18, 2026
added 7 commits
August 18, 2026 17:31
…iew) The test gate before the deploys is gone: the full suite runs on every PR, each deploy job runs its own pnpm build (a broken build cannot reach wrangler), and what a master push needs verified is production — which only exists after the deploy. The @smoke e2e subset is that verification, and the weekly canary covers the rest. Re-running unit on every merge bought nothing the PR run had not already proven. The compensating control for merge skew (a PR tested before master moved) is branch protection's 'require branches to be up to date' — a repo setting, recommended in the PR body, not a workflow job.
The wire capture is un-awaited and its json read best-effort — if it lost the race, a demo existed (the dialog was showing its link) while afterEach saw null and skipped the revoke. The dialog link's id is now the recovery path: demoId ??= linkId before any assertion can throw, and the two sources are asserted to agree when both answered.
This was referenced Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Last in the DEV-2203 stack — now also carries the deploy-first master pipeline (Dan's #184 post-merge review). Stacked on #187; merge the remaining chain bottom-up (#185 → #186 → #187 → this).
Master pipeline: deploy first, verify production after
pnpm build(a broken build never reaches wrangler); what a master push needs verified is production, which only exists after the deploy — the@smokesubset (one render per engine, share fixture, docs example, one Style round-trip, ~1 container) is that verification.master. That makes PR CI results exact for the merged tree and is the compensating control for merge skew now that master doesn't re-run the suite.The rest (unchanged from previous revisions)
e2e-live.ymlrewrite: manual dispatch (inputsbase_url,ai,pkg_pr_new_ref), weekly Monday 05:00 UTC prod canary (AI on),workflow_callsmoke mode; folds in the three orphaned live specs; container steps strictly--workers=1; the authed-share step turns a rottenE2E_BROKER_TOKENinto a::warning, never a red run, and scrubs traces so the JWT can't reach public artifacts.Notes
ai: true.Note
Medium Risk
Changes production deploy gating (no pre-deploy full CI on master) and adds scheduled live E2E against prod and real containers; mitigated by PR CI, deploy self-builds, and a focused post-deploy smoke subset.
Overview
Master pushes no longer re-run the full PR CI suite.
master.ymldrops thetestjob that calledci.yml; deploy jobs only needchanges, and a newsmokejob callse2e-live.ymlwithsmoke: trueagainst prod after at least one deploy succeeds. Merge-skew is expected to be handled by branch protection (“require branches to be up to date”), not a second full suite on master.e2e-live.ymlbecomes the hub for live E2E: weekly Monday prod canary (AI on),workflow_callfor post-deploy@smoke, and richer manual dispatch (ai,pkg_pr_new_ref). Full runs split into parallel Sandpack/API-read specs vs container suites at--workers=1, plus optional broker-gated authed share (token validation → skip/warn, no traces on that step) and AI checks.e2e-starter-matrix.ymladds a monthly cron (1st at 03:00 UTC) with prod as defaultE2E_BASE_URLwhen inputs are empty.share-create-live.spec.tsrecovers the minted demo id from the dialog client link when the POST response listener misses, soafterEachrevoke still runs.Docs in
ci.yml,AGENTS.md, and workflow comments are updated to match deploy-first + PR-only full CI.Reviewed by Cursor Bugbot for commit 1c5c441. Bugbot is set up for automated code reviews on this repo. Configure here.